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ABSTRACT 

The purpose of this paper is to select from among the 
bodies of research on complex skill learning those generalizations 
that seem to have some applicability to issues in designing training 
materials for teacher education. That selection entails a review of 
the principal points and findings within the domain and of the 
implications of the findings for the development of training 
materials for teacher education. The first section of the paper 
delineates the characteristics of skilled performance and briefly 
examines the stages that the learners pass through as they acquire 
such skills. Three variables are identified — instructions, practice, 
ard knowledge of results--t hat have been widely investigated and that 
are applicable to training materials. Summarized are some of the 
principal generalizations about these variables^ especially as they 
seem to pertain to the ultimate objective of designing training 
materials. The second section of the paper deals with an analysis of 
skilled perfornance in teaching and extrapolates from the research on 
complex skills learning how one might design training materials in 
the area of the teaching of concepts. (Author/DDO) 
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FOREWORD 



!n n previous article, Professor Hudgins investigated the research on concept acquisition and from that literature 
constructed a well-reasoned procedure for the development of protocol m«Uei':als. Using a parallel stratenv. Professor 
Hudgins has surveyed the research on complex skill learning to see if it is pos5ibl<i to extrapolate from that research mean- 
ingful guidelines to developers vvho are interested in materials which lead to the acquisition of teaching skills. 

In pursuing this task three categories of variables in skill acquisition are d^'-ined which then serve as the primary foci 
around which generalizations are extracted from the research. Those categories are (a) instructional variables, (b) practice 
variables, and (c) feedback (knowledge of results) variables. An illustrative skill, is then ana^^yzed in terms of a series of 
sub-skills derived from research on that particular skill. In this instance the teaching skiM chosen for analysis is that of 
the "teaching of concepts/' At this point. Professor Hudgins conjoins the two analytic effort,s and attempts to apply the 
generalizations gloaned from the research on complex skills and the research on the teaching of concepts to produce prop- 
ositions about the elements of teacher training materials and something of the character of those elements. 

Perhaps one of the more interesting contributions of the authoc may be his concept of "iolf nontained teacher 
training materials." Professor Hudgins argues initially that skill in teaching anything must be based upon an empirically 
grounded model of that teaching skill. In this case, the model would be one of the "teaching of concepts." This model 
must be taught and learned as a kind of cognitive phase of skill development (perhaps the words "protocol phase" 
could be substituted for "cognitive phase"). The second phasu involves the "self-contained training materials." Materials 
in this phase include opportunity for extended practice with feedback in a way that does not involve all the complexities 
of the actual field context. Instead the practice entails a minimum of external support in the sense of equipment, 
personnel, etc. It strives for efficiency in practice and feedback conditions rather than realism, an interesting and per- 
haps controversial idea. 

The final part of tho paper takes the concepts of "longitude" and "latitude" and outlines in general the form which 
the training materials would take with respect to the instructions to be given the teacher trainee, the conditions of prac- 
tice, and the kind of feedback which would be appropriate. 

In my opinion, the article constitutes not only an interesting set of guidelines to the developer but also a model of 
the uses to which the research literature can be put in the development of such guidelines. 



L. D. Brown, Editor 



Self-Contaitied Training Materials for 
Teacher EducaCion: A Derivation 
from Research on che 
Learning of Complex Skills 

Bryce B, Hudgins 
Washington University 

The purpose of this paper is to select from among the bodies of re- 
search on complex skill learning those generalizations that seem to have 
some applicability to issues in designing training materials for teacher 
education. That selection entails a review of the principal points and 
findings within the domain, and implications of the findings for the de- 
velopment of training materials for teacher education. The paper is 
structured as follows. In the first section, we begin by delineating 
the characteristics of skilled performance and briefly examine the stages 
that learners pass through as they acquire such skills. Then, we iden^ 
tify three variables — instructions, practice and knowledge of results — 
that have been widely investigated and which are applicable to training 
materials. We summarize some of the principal generalizations about these 
variables, especially as they seem to pertain to our more ultimate objec- 
tive of designing teacher training materials. The second section of the 
paper is concerned with an analysis of skilled performance in teaching 
and some admitted extrapolations from the research on complex skills 
learning to how one might design training materials for teacher education. 
In an effort to produce discussion at a level commensurate with the pro- 
blems of such development, we shall try to extrapolate the principles of 



skill learning to the is5iues of designing Craining materials in the areci 
of the teacliing of concepts. 

Tlie Dimensions and Learning of Complex Skills. 

Characteristics of Skilled Performance 

What characterizes the performance of an individual who possesses 
a skill in great abundance? A description will help us to see what kinds 
of criteria we want to impose upon the design of training materials in 
order that they may lead to appropriate outcomes . The game performance 
of an accomplished athlete makes a good beginning point for this analysis. 
On a baseball team, for example, the most skilled fielder is commonly 
placed at the shortstop position. On a ground ball hit hard to the right 
of the shortstop into the deep position, or the "hole", as it is called, 
the player must move at a sprint from his normal placement in the direc- 
tion opposite to that where he must throw' (first base). Ordinarily, when 
he grasps the ball, his momentum is carrying him away from the direction 
of his throw. He must turn, off balance, and throw forcefully if he is 
to beat the runner to first base with the ball. Sometimes, the throw is 
made while the player is off the ground, turning in mid-air to face in 
the direction that he throws. Of course, this performance is only exe- 
cuted successfully by the best players, and not always by them. 

If we examine records of such performap-^as , or more mundane ones, 
certain characteristics of the skilled performance emerge. For one thing, 
Che t iming of the performance is good. Whatever the demands of the per- 
formance or some element of it, the performer seems to have plenty of time 
to do what he wishes to do, or must do. Some years ago, my wife and I 
Joined a square dance group. The initiation into this organization in- 



volved a series of lessons, some eight or perhaps ten of them, one even- 
ing a week. If you have ever square danced, you know that the one thing 
you do very little is to stand around. Square dancers seem always to be 
on their way from one place to another and, of course, they have a precise 
number of measures to arrive at the proper next station. Ther^^ are many 
ways to distinguish between a novice and a veteran square dancer, but I 
have found the most reliable one is to assess where each is, in relation 
to his destination, on the beat before the last beat. You will invariably 
find the accomplished dancer one rather jaunty stride from where he is 
heading, and the beginner either standing around sheepishly where he was 
headed for (or thought he should be headed for) or a mile down the road, 
coming. hell bent for election. Let me anticipate the discussion a bit 
here and indicate that one of the things the beginner most requires is a 
cognitive map of the skill. As I say, we shall return to that idea shortly. 

The skilled performer also has an advantage over the beginner in that 
he can anticipate what is to come. With proficient typists or telegraphers 
this amounts to the ability to group larger chunks of information before 
beginning to type or to transmit. Sometimes this is referred to as the 
ability to "copy behind." In square dancing, to continue that metaphor, 
the veteran dancer has in mind that after the present maneuver, the dance 
will shift to an "allemande left.'' A firm knowledge, not only of the seg- 
ment he is now dancing but also of the position of his body, arms, and 
hands when he hits the corner, permits him to swing his partner with vigor 
and come around, arm bent just so, headed down the line, set for the alle- 
mande. To the beginner, on the other h-^nd, each movement is new, not 
quite learned. It st.i 11 represents a s igni f iCctnt challenge to him just 
to get through the "now" moment, never mind the fancy footwork that lies 



ahead. 

The skillful performer has long since eliminated false moves, unnec- 
essary little steps and starts that impede, if not the actual effectiveness 
of his performance, its stylistic or elegant nuances. This accounts in 
part for his ability to be in the right place at the right time and to 
anticipate the next set of events. Finally, most skilled performances be- 
come aja_toma^_ic or autonomous. Thus, the typist fully habituated to dealing 
with mounds of manuscript copy may pound happily away at the day's work, 
averaging eighty or ninety words a minute over long stretches of hours, 
and with only an occasional error. Such a performance is enough to astound 
the beginning high school student of typing with his low speed and seem- 
ingly endless string of mistakes. However, even the expert suddenly en- 
counters not the neat rows of manuscript copy but, let us say, tables of 
detailed data, tables that demand the most exact duplication lest their 
meaning be altered. It is at this point that the responses of the expert 
are challenged. No longer can the typist think about her menu for that 
evening, or what dress she will wear to the weekend party. As the diffi- 
culty level of the task increases, her attention to the task is demanded. 
Similarly, to bring the point home, you may be driving along the highway, 
responding automatically tu the fairly constant stimulus conditions of 
the road, signals, light and weather conditions, when suddenly a large, 
slow moving truck pulls from a side road immediately into the path of your 
car. At once, the automatic quality of your driving is replaced by in- 
tense attention to the details of the situation. You may have to call 
upon all the skill your years of experience as a driver have provided you 
with to avoid a serious accident. Again, what differentiates the skilled 
performer from the novice is that, in critical situations, the skilled 



performer is able to bring his capabilities to bear upon the situation, 
to avoid an accident, to throw out a speedy runner, to contribute to the 
graceful flow of a square dance, or to teach a concept in a situation 
that presents unusual aspects. 

Fitts (1962), in his analysis of complex skills, breaks their learn- 
ing into a series that, while it is similar to what we have just said, 
also adds a dimension that is particularly pertinent for people involved 
in producing training materials that are more clearly verbal and cogni- 
tive in composition than they are simply motoric or even perceptual. 
Thus, Fitts says that three overlapping stages occur as complex skills 
are mastered. First is a cognitive phase, followed by a phase of fixa- 
tion , and finally an autonomous phase. The last two are similar to our 
previous discussion. Most important, however, is Fitts' identification 
of a cognitive phase in the learning of complex skills. In short, this 
means that in the beginning what the learner requires is a cognitive map 
of the skill he is to learn how to perform. This would include some 
specification, when pertinent, of the purposes of the skill, what he will 
accomplish by performing it. For example, with respect to materials for 
teacher education, you might wish to communicate to the student the re- 
lationship between the skill to be learned, and other more complex skills 
or how this skill will enable him to overcome a traditionally difficult 
phase of teaching, or something of the sort. 

But more directly, the cognitive phase involves helping the learner 
to specify the elements of the performance, what sequence is demanded or 
what options may exist with respect to sequences, and, in general, the 
nature of the final performance. In a very real sense, this is a time 
when the learning of the student can be facilitated by assisting him to 



form a concept of what is contained in the skill, how its elements fit 
together, and how his present knowledge and experience can contribute 
(tha. is, transfer positively) to what he is to learn. 

Instruct ions 

When we want someone to learn a new skill, naturally one of the first 
things that occurs to us is to provide some kind of instructions for the 
learner. At one level, these instructions may be designed to communicate 
to the learner what the task is and what the objective of the learner must 
be if he is to perform it successfully. For example, we may say to the 
adolescent in the first stages of driver training, "Steer the wheel of the 
trainer so you always stay on the road and in your righthand lane. If you 
go off the road or in the wrong lane, a buzzer will sound. Your job is to 
keep the buzzer turned off as much as possible." 

Such a direction, sometimes called a set or expectancy, gives the 
learner a simple conception of what the task consists of, or of what he 
must do in order to be successful in executing the task. It would be very 
difficult to prove, but we would suppose that the more complex the task, 
the more valuable such a cognitive orientation would be. Fitts (1962), 
who, perhaps more than most invei^tigators , has analyzed the dimensions 
of complex tasks, reports that orientations for beginning pilots that per- 
mitted them to "intellectualize" the cognitive components of the task of 
flying resulted in solo flying occuring earlier than for another group of 
of student pilots who were not given instructions that offered the same 
opportunity. 

Ausubel (1968) has stressed the importance of emphasizing the poten- 
tial meaningfulness of verbal material. When the organization and struc- 
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ture of material, such as that included in CexCbooks or lectures, is ig- 
nored, teachers and students alike frequently treat the task as meaningless 
and approach it as though the material had to be learned by rote. Although 
it is an extension of Ausubel's point, the application to the learning of 
skills seems s traightf orvv'ard . If the overall strategy of skill performance 
can be made clear to the learner in advance of preliminary efforts at prac^ 
ticing the skill, it should be possible for him to achieve maximum positive 
transfer from what he already knows that is in any sense pertinenc co the 
new skill to be learned. In our previous illustration of driver education, 
virtually all children have had extensive practice at steering other simple 
vehicles such as bicycles and sleds, and they possess at least rudimentary 
knovv'ledge about the rules and customs of traffic behavior in our culture. 
In so simple a case, much positive transfer probably occurs almost auto- 
matically, but when the skill to be learned is more complex, great advan- 
tage can accrue if the trainer provides instructions that assist the 
learner to see the relationships between what he already knows or is able 
to do and the task that currently confronts him. 

The designer of training materials often wishes to insert instructions 
within the task. The effectiveness of such instructions depends upon sev- 
eral factors. Verbal instructions may be very helpful if the task is pri- 
marily verbal. If, however, as is true of many complex skills, the task 
is heavily perceptual or motoric in nature, verbal instructions may add 
little, and may actually interfere with the process of learning. Individ- 
uals differ, also, in the medium of instruction that they are capable of 
being helped by. For many people, instructions that are embedded in an 
activity instead of being given in directly verbal form are more helpful. 
Finally, and perhaps of most significance to the present discussion, is 



the fact that the simpler the inscructions provided are, the more helpful 
they are likely to be. If the trainer attempts to build into the midst 
of his materials a theoretical or conceptual background for learning the 
task, he and his materials are in deep trouble. In such cases, a simple 
verbal instruction or prompt that helps the learner to overcome a rough 
spot in the learning is much more effective. 

Hilgard and Bower (1966) , in a discussion of training in "trouble- 
shooting" (diagnosing and solving problems with electronics equipment) , 
make a similar observation. They report that in one study trainees were 
provided with situations to diagnose and were given various possible solu-- 
tions. They were to identify the best diagnosis, which was then compared 
with pre-programmed diagnoses offered by experts in the field. The objec- 
tive of such training was to teach the trainees *'to think like the experts." 
Interestingly, no formal instruction was given to the learners in the 
theories or concepts of trouble-shooting, although these abound. As Hilgard 
and Bower comment, "In view of various theories of trouble-shooting. .. it 
is a bit disconcerting to have the rather blind 'apprentice' method work 
so well, though of course the optimal sequence may have had all the nec- 
essary 'theory' built into it," (Hilgard and Bower, 1966, pp. 548-549). 

The point was well made by an early study of motor tracking (Renshaw 
and Postle, 1928). Subjects who were simply told to keep the apparatus 
running were designated "uninstructed." "Instructed" subjects were bur- 
dened with a detailed set of instructions- For the learner, such instruc- 
tions can entail an excessive informational load, and verbal directions 
have to be translated into instructions to oneself about how to perform. 
With succeeding days of practice, the error rate of the uninstructed sub- 
jects diminished far faster than that for subjects given the detailed 
instructions . 
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The Effects of Practice 

"Practice makes perfect" is one of the most widely knoim pr escientif ic 
generalizations about the improvement of skills. Although the generaliza- 
tion needs modification here and there, certainly there is evidence on all 
hands that the characteristics of skilled performance that we presented 
earlier do not emerge unless large amounts of practice with the skill are 
undertaken. During recent years, the world has witnessed both the Winter 
and the Summer Olympic Games. Thanks to television, we were able to ob- 
serve first hand the performances of the world ^s most outstanding athletes. 
Time after time interviews \^ith these virtuoso performers revealed that 
they practice their arts or skills perhaps from three to five or six hours 
each day. Furthermore, for most of these young men and women, that regime 
of practice has gone on since they were small children. We might suppose 
that the world's champion figure skaters and platform divers, to pick 
examples, have devoted as much as fifteen thousand hours to the practice 
of those skills. It is difficult to visualize what fifteen thousand hours 
means. One quick translation is to point out that a child spends about 
that much time in school from the day his mother registers him in Kinder- 
garten until he receives his diploma from the local high school. How 
practiice affects the performance of a complex task can be seen in Figure 1 
on page 10. When subjects practiced mirror drawing once a day, their skill 
improved regularly over a period of three months. The most impressive 
example that I have located of how p.erformance continues to improve with 
practice is ^o be seen in Figure 2 on page 11, 

Few laboratory studies have the capability of following the course of 
practice over many extended trials or long periods of time. The data shown 
in the figure comes from an industrial study. What it reveals is that skill 
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Figure 1: Improvement in mirrar drav/ing as a function 
of daily practice. (Reproduced from Deese 
and Hulse, 1967, p. 453.) 
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Figure 2: Improvement in an industrial task with 
repeated practice. (Reproduced with 
permission from Holding - "Principles 
of Training" (1965), Pergamon Press Ltd. 



In cigar making continues to show improvements through approximately three 
million trials spread over a full two year period on the job. Nor is that 
an isolated finding, ^^^-len industrial or educational studies have examined 
performance during extended periods of time, involving many, many trials, 
decrements in time to perform an operation, or in the number of errors 
made, both of which are traditional indices of improved skill, tend to show 
up. I suspect, although I have no evidence to submit on the point, that 
most skills of teaching that we wish students to learn receive absurdly 
small amounts of practice, far underestimating even reasonable amounts to 
develop and fix skills at passable levels. One hypothesis I would surely 
suggest for your examination is that the design of training materials for 
education of teachers should include estimates of the amount of practice 
that is likely to be necessary to attain the required level and smoothness 
of the skill. It would then follow, of course, that the design of the 
materials should include any conditions that are necessary to bring about 
the required amount of practice. 

One of the earliest observations pertaining to the role of practice 
in skill learning was the famous study by Bryan and Harter (1897) about 
learning telegraphy. That was the study that gave us the concept of 
plateaus in learning. Actually, the plateau that occur ed there was an 
artifact of the conditions, or a change in conditions of practice. Later 
studies of skill and practice have not usually revealed plateaus at all. 
Of course, whenever we consider the practice records of an individual 
learner, not smoothed either over repeated performances or by groups of 
learners, we do not find the steady kind of increment in performance of 
the cigar making curve. Individual records tend to be quite erratic, but 
they do show improvement over time if regular amounts of practice continue. 



Similarly, we have gradually come to recognize that earlier concepts of 
motivational and physiological limits of performance are artifacts of par- 
ticular experimental arrangements." 

Knowledge of Results 

Perhaps no variable plays a more important role in training on com- 
plex skills than that of knowledge of results (Biiodeau and Bilodeau, 1969). 
Put simply, knowledge of results is information given to the performer 
about the effectiveness of his performance. In general, psychologists 
have foind that performance improves when the knowledge fed back to the 
actor has a controlling or directive function over subsequent trials. For 
example, skills such as target shooting or hitting a golf ball can provide 
information to the learner about the relation^ship between his aim, or his 
swing, and where the bullet or the ball went. 

Let us begin the discussion of knowledge of results (KR) with a brief 
review of one of the ancient but definitive demonstrations of the point. 
In the 1920 's, Thorndike (1927) had people draw lines of 4, 5, or 6 inch 
lengths while they were blindfolded. Half of these subjects were given 
no information about their line drawing skill, but the other half were 
told ''right" if the line was within a quarter inch of the criterion, or 
**wrong" if it was not. Not surprisingly, the line-drawing performances 
of the group given information or knowledge of results improved, but that 
of the other group did not (although the performance of that group sta- 
bilized, which is frequently overlooked). A few years later, Trowbridge 
and Cason (1932) refined and extended the original investigation, the 
results of which are summarized in Figure 3 on the next page. 
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Figure 3: Mean errors in line drawing by subjects 
given different kinds of knowledge of 
results. (K. R.) (Reproduced with per- 
mission from Holding ^ "Principles of 
Training" (1965), Pergamon Press Ltd.) 



The control group in the figure was the same as Thorndike's group 
\^it[\ no KIl, while the group labeled R-W was given the same kind of feed- 
back about their performance as his KR group. Trowbridge and Cason, 
however, did two things differently than Thorndike. For one group, not 
only was KR provided, but it indicated in detail how much too long or too 
short the line was. You can see from the figure that this kind of know- 
ledge reduced the number of errors that people made. The group labeled 
"Nonsense" was another control group, given not substantive feedback 
about performance, but simply a nonsense syllable response after each 
trial. This was done to control for the possibility that any kind of 
verbal response after an attempt would reduce errors. Since the results 
for that group were no better than the control subjects (in fact, some- 
what worse) the possibility can be dismissed. These studies and others 
similar to them have demonstrated the value, even the necessity, of know- 
ledge of results for the improvement of skilled performance. But the 
issue is a complex one. Many motor skills provide a type of feedback 
as the result of a chain or sequence of events and, while the information 
may be of a generally instructive nature, often it provides little spec- 
ific information about how the performance of the skill shoul.d be altered 
if it is to be improved. Consider a relatively simple performance such 
as basketball free throws. If the ball does not flow through the hoop, 
a certain amount of information is available from where it does land. If 
it fails to reach the basket, throw harder. If it hits the front of the 
rim, try a higher arc. If it bounces off the backboard, use less force, 
and so on* This general kind of information coupled with hours of count- 
less school boys through the years. But how much greater could the im- 
provement achieved be if the learner received feedback about each element 
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in hiii performance? Suppose, for example, a buzzer sounded each cime Che 
boy gripped Che ball incorrectly, held ic Coo firmly or Coo loosely, chrusC 
it backward coo far before shooting, etc.? We may leave it to the physical 
education department and the baskecball coaches Co work ouc Che Cechnology 
for providing such feedback to their players, but the point has wide appli- 
cability with respecc Co complex skills. 

Psychologiscs frequently Calk abouc intrinsic feedback and artificial 
(sometimes augmenCed) feedback. Artificial feedback can help improve per- 
formance quickly. The difficulty is thac unless the augmented feedback 
assists che learner in developing his responses on the basis of the intrin- 
sic elements of the cask, augmented feedback has litcle residual benefit. 
For example, in a study of aircraft gunnery, it was learned that gunnery 
students had difficulty "framing*' the target, this is, setting the correct 
range for firing. A device that provided an audible click when the proper 
range was established improved the performance markedly, but when che arti- 
ficial feedback (the click) was removed, performance dropped off immediate- 
ly to what it had been for a control group with the same amount of practice 
l>nien workers are learning to move unfired potcery, they must not seize it 
too tighcly. In another study, it was found chat the correct pressure can 
be learned if a signal comes on whenever the grasp is too forceful and 
goes off when the pressure is diminished appropriately. Once an appropri- 
ate range of pressures has been learned by the worker, the augmented feed- 
back can be eliminated with no perceptible increase in the number of pots 
damaged by apprentice workers. Apparently, Che worker picks up sufficient 
proprioceptive cues associated with "signal on" and with "signal off" con- 
dicions Co behave independently of such artificial cues after a period of 
learning. 
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The critical point, of course, is not whether augmented feedback is 
advantageous, for its use is indicated with many sorts of complex skill6'. ; 
rather, the problem that confronts the trainer is how to guarantee that 
the learner will integrate the appropriate responses with intrinsic ele-- 
ments of the task. This would seem to be a vastly important observation 
for producers of training materials in teacher education. 

Finally, we must raise the question of when feedback or knowledge 
of results should be administered. There are two fundamental conditions 
under which this might happen. First, feedback may be given as the skill 
is being performed. The previous examples of signals on and off during 
"framing" or moving pottery involved such concurrent feedback. Alterna- 
tively, the training material might not provide feedback until the end 
of the performance. In this connection, does it matter whether the feed- 
back is immediate or delayed? 

Concurrent feedback is important for the action elements of perfor- 
mance, that is, for matching performance to the standards set for the 
learner. But, for relatively simple tasks at least, learning is more 
influenced by knowledge of results that succeeds the performance. This 
finding is not independent of the necessity that the learner base his 
performance upon cues intrinsic to the training task. 

Whether or not knowledge of results must be administered immediately 
after the completion of the performance is a more critical question for 
studies with lower order animals than it is for training human beings. 
There is evidence that even slight delays (of 30 seconds or less) in pro- 
viding reinforcement to animals after performing a task sharply reduce 
the amount of learning. However, with human beings, particularly when 
task performance can be assessed through verbal feedback, brief time de- 



lays are of no consequence, and even longer ones, from a day to a week, 
have been shown not to interfere with learning. That is, verbal feed- 
back can be utilized by human learners to reinstate the original perfor- 
mance, or the learner's retained image of it, and to relate what he was 
doing with the knowledge of results about it. It does not seem advisable 
though, to permit the subject to repeat the performance of a skill before 
he has received knowledge of results about his first trial behavior. 

A Proposal for the Development of Self-Contained 
Training Materials in Teacher Education 

The relevance and applicability to the learning of skills of the 
concepts of instruction or directions, practice and feedback or knowledge 
of results has been presented in the first portion of this paper. We 
shall attempt to organize the second section of the paper around these 
same topics but modify our discussion of them to fit the special demands 
of designing training materials in teacher education. As an informal 
test of the adequacy of these topics as generalizations, we shall try 
to illustrate how a particular set of teacher training materials might 
be constructed, and for that purpose we have chosen the skill of teach- 
ing concepts. No skill is more ubiquitous in teaching. It is probably 
safe to say that in better than half of their classroom verbal inter- 
actions with students, teachers are attempting to develop concepts in a 
specific subject area. Concepts such as ^'subject" and "predicate" in 
English grammar, "more or less than" in arithmetic, "deciduous" and "ever 
green" in botany are examples. The fact that teachers do not always 
clearly label or specifically define the concepts they teach does not 
lessen the generality or significance of this activity in teaching. The 
fact that they do not always teach concepts effectively or efficiently. 
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on the other hand, is the occasion for us to deal with the design of 
teacher training materials in this area. First, however, we must give 
some attention to the nature of concepts themselves and to a model for 
teaching them. 

The Nature of Concepts in Relation to Teaching . 

A concept is an abstraction based upon the common properties or 
elements possessed by all the members of a class , but not exclusively 
descriptive of any single member of that class. Words are labels for 
concepts, and definitions are an attempt at the level of logic to iden- 
tify the genus and differentia of the class. 

To teach a concept involves some skills that are unique and others 
that are common to more general issues in teaching. Among the latter 
we would include such items as assessing the knowledge of learners be- 
fore and after instruction, and of seizing upon whatever feedback cues 
are available in the situation to make instruction contingent upon the 
comprehension and pacing of the learners. 

Skills that are unique to the teaching of concepts would include, 
at a minimum> the ability to identify concepts and to discriminate those 
points in instruction at which teaching a concept is the appropriate 
strategy. Furthermore, the specific task of concept teaching demands 
that the basic attributes of the concept be explicated, including its 
limiting cases or conditions. Tactics that would frequently be used to 
teach concepts, either altogether or in varying combinations, include 
illustrating positive and negative examples of the concept, defining the 
concept, and differentiating between the concept in question and others 
of a similar nature or similar in one or more particulars with which the 
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target concept might be confused. With this very brief introduction to 
the teaching of concepts, what Implications can we find in the research 
on complex skill learning for the design of training materials in the 
teaching of concepts? 

Some Assumptions About Transfer, and a Model for Teaching Concepts . 

It is important to note at the outset of this discussion that some 
additional ideas are required to clarify and to complete our illustra- 
tive case. In the first place, most skills that we want teachers to 
master have a conceptual basis. This is certainly true of teaching con- 
cepts* For, no matter how many different concepts we choose to incor- 
porate in training materials for teachers, we will never exhaust those 
that a teacher might be called upon to teach. We assume that a large 
amount of positive transfer occurs between the training materials that 
we provide and any particular concept that the trainee later is called 
upon to teach. Indeed, if we do not make such an assumption, it is 
difficult to see what benefit would derive from engaging in training. 

The theory of transfer of training to which we appeal here is that 
often called '^transfer of principles," "nonspecific transfer," or "gen- 
eralization." Perhaps we would do well to identify it as a "transfer 
of model," for what we would do is to train the teacher to make a var- 
iety of complex but sequentially dependent responses that conform, on 
the whole, to the model of the teaching of concepts that we use to plan 
our training materials. We cannot overemphasize the point that the 
validity of our training procedures , and their outcomes in educational 
practice, will be jeopardized to the extent that the models we select 
are of dubious validity. This fact argues for a continuing and healthy 
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linkage between research on problems of teaching and the development of 
training materials, an obvious but usually overlooked problem. In the 
case of the teaching of concepts, the work of Clark (1971) provides a 
model for the classroom teaching of concepts. Clark's model is induced 
from his assessment and interpretation of a large number of experimental 
studies of concept learning. Unfortunately, few of these studies had 
their origin in classrooms, and the model developed by Clark is only one 
of a number that might be developed, and perhaps not the one most perti- 
nent for the training of teachers. The fact that this is so, in a field 
as carefully studied as the learning of concepts, intensifies the need 
mentioned earlier for a careful articulation between research and the 
development of training materials. The first point in our argument, then, 
is that each concept with which trainees practice can be regarded as a 
learning trial. What is learned, ultimately, is not simply responses to 
be made to each of a theoretically large number of trials, but a general 
model for teaching concepts, one which would obviously have to be modi- 
fied in particular (and therefore unpredictable ways) from trial to 
trial, or from one teaching situation to another. 

The Clark model for the teaching of concepts includes several ele- 
ments . The first and last of these are applicable to teaching episodes 
of any kind, namely that the teacher should in the first place assess 
the preiustructional knowledge of his pupil and that, finally,, there 
should be an evaluation of how adequately the concept has been learned 
by the pupil. Between these two assessments, there are ba.^ically two 
components of instruction: an introductory one and a confirmatory one. 
Both of them are, however, to be guided by some general precepts having 
to do with offering examples or instances to the learner in a clear, 
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unhurried and systematic manner. In the introductory stage, all instances 
should be positive examples of the concept and the critical or criterial 
characteristics of the examples should be enumerated and explained. The 
confirmatory process involves gradually mixing negative instances (non- 
examples) Into the sequence of examples and allowing the learner to iden- 
tify each according to whether it does or does not exemplify the concept. 
This should be accompanied by complete feedback after each response and 
with due provision for the learner to consolidate his response to one 
instance before the next one is presented. 

Self -Contained Training Materials and th eir Place in the Teacher Training 
Sequence . 

Such a model can, of course, be learned by the teacher at several 
stages that vary on a dimension of classroom or teaching reality. A first 
stage would be similar to what we have already presented; that is, the 
teacher could cognize the model in an abstract sense, much as he might 
learn it from the study of a textbook, A final stage, representing some- 
thing closer to the concept of mastery that we are after, would have the 
teacher applying the model in his instruction of real pupils in real 
classroom settings. Such a stage might be represented in the teaching 
done by a student teacher or by an experienced teacher in his daily in- 
structional activity. These stages are shown in Figure 4, which also 
includes Stages 2 and 2a, on page 23. 

Stage 2 introduces the notion of self-contained training materials. 
Self-contained training materials are those materials that allow the 
learner to practice his skill without reliance upon external agents, such 
as the availability of a group of students to teach, the presence of a 
supervisor or other human feedback mechanism, and so forth. We would 
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Stage 1 



Stage 2 



Stage 3 



Abstract model; 
practice coincidental. 



Self-contained, model- 
based training materials. 
Set, practice and feed- 
back controlled by model. 



Context-based practice. 
Live training; contingencies 
set by conditions of class- 
room practice. Model becomes 
subordinate to practical 
exigencies. 

A 



Stage 2a 

Knowledge base Elements of training, particularly 

of model. practice and feedback, controlled 

by data fed into training materials. 



Figure 4: Self-Contained Training Materials in a Teacher Training Sequence. 

The diagram shows (1.) the dependence of training materials upon 
an adequate knowledge base for the model and (2.) the position 
of the materials as the reality dimension of training is varied. 
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wish to hedge this definition to the extent of excluding from the cate- 
gory "external agents" such equipment: as computer terminals or machinery 
associated with programmed instruction, for the benefits to be gained 
from using such equipment may be sufficiently great to offset the nec- 
essary demands on space and physical housing. But the basic sense in 
which we use the term "sel f-contained" is no different from the simple 
arrangements under which one can ordinarily practice playing a musical 
instrument or work on a variety of motor skills. The pianist needs only 
his piano, and perhaps his music, to be in business. The basketball 
player can practice free throws for hours on end, given only access to 
a basketball and a standard and hoop through which to throw it. 

It is possible to provide training materials with the same advan- 
tages for teachers, if certain conditions are met. These conditions may 
be of two kinds, represented in the figure by Stage 2 and Stage 2a res- 
pectively. To provide practice at Stage 2, several fundamental elements 
are required. These are, first of all, a model of the skill to be learn- 
ed. For the teaching of concepts, the model of Clark is illustrative. 
Again we must emphasize that the base of knowledge upon which such models 
are constructed is of paramount significance. This is true because both 
the responses that are called for by the tiaining materials and the kind 
of feedback that is built into them must be derived directly from the 
model. That is, the contingencies of practice and of feedback are model- 
based. If the model does not provide skill development that is congru- 
ent with the way the skill must function in real situations, negative 
rather than positive transfer may well result. 
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Secondly, the training materials themselves must be prepared in ac- 
cordance with the elements of the model. One does not teach concepts in 
general, and the training given to the teacher trainee must indicate a 
specific concept to be taught. It is incumbent upon the developer to 
analyze each concept and to prepare feedback that will be meaningful to 
the trainee. In the case of our illustration of the teaching of concepts, 
for example, the developer would wish to write out a series of illustra- 
tive positive examples of each concept which the trainee could use for 
purposes of comparison with his own examples of the concept. 

The necessary condition for materials developed at Stage 2a is that 
such materials be based upon direct experience by the developer or his 
staff in teaching the concept. Such experience is not intended to take 
the place of research on the model of the skill and it does not refer to 
the base of knowledge upon which the model is built. Rather, the purpose 
would be to generate some actual data obtained from children when the 
model is applied to" teaching them the skill. These data would take the 
form of feedback to the trainee. 

To summarize (before we finally turn to a more detailed examination 
of the roles of instructions, practice and feedback in training materials 
for teaching skills) , I believe that the concept of self-contained train- 
ing materials has two great advantages to recommend it. The first of 
these is the accessibility to the trainee of extended practice on the 
skill in question. Some dozen years ago, Shaplin (1961) observed that 
teachers have relatively little opportunity to engage in the directed 
practice of teaching. When one asks the question, "How much opportunity 
does a teacher have to practice a skill such as teaching concepts under 
circumstances that are aimed at helping him to improve the skill?," the 
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answer seems to be that the opportunity is quite limited, indeed. Self- 
contained training materials clearly would allow each trainee to practice 
to Che point of mastering the skill. In the same way, the materials make 
allowance for individual differences. One's progress is determined only 
by the amount of practice one requires. Such arrangements clearly make 
it possible for teacher education programs to begin to individualize in- 
struction for their trainees, and to remove some of the lockstep elements 
of classroom instruction — a condition which these same trainees are 
continually admonished to provide for their own pupils but for which they 
are almost never provided models as part of their own student experience. 

The second great advantage afforded by such self-contained training 
materials is the occasion they provide for teachers to learn generalized 
schemes, concepts or models for the skills they are teaching. The pro- 
bability that teachers will modify their behavior to avoid slavish appli- 
cation of formalistic teaching rises more when they possess the broad 
image of a skill that is possible from a generalized scheme of it than 
when what they have learned is merely a set of sequenced behaviors with- 
out such a cognitive map. We would add a third advantage, although it 
may be of less direct significance to the developer of training materials 
and to the trainees with which they are to be used. This is the hope 
that the development of models of skills will be carefully accompanied 
by checks on the knowledge bass underlying the model and, where required, 
that appropriate kinds of research studies will be undertaken to support 
or to modify ongoing development. As an example of what is intended here, 
1 would point to the work that Gage and his associates accomplished at 
the Stanford Center for Research and Development in Teaching in their 
studies of the ability of teachers to explain subject matter. Several 
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years were devoted to carefully designed experimental studies of the var- 
iables that govern explaining behavior. This work was ultimately trans- 
lated into a training program that conforms in part to the description 
that we have given here of self-contained training materials. 

Designing Self-Contained Teacher Training Materials . 

Let us move now to the application of principles established in the 
first section of this paper to the problem of developing training mater- 
ials for teacher education. One issue that becomes immediately apparent 
Is that training materials must be specific, not general. Our proposal 
for training teachers to teach concepts must include practice in the teach- 
ing of real and specific concepts, even though its larger objective is to 
contribute to the development of a broad set of skills that improve con- 
cept teaching (and pupils* learning of concepts). Posner and Keele (1973)5 
in a recent major review of skill learning, indicate that their orienta- 
tion to the improvement of skills through training is to analyse the com- 
ponents of specific skills and to concentrate upon improving performance 
in the most difficult components. 

Therefore, we shall consider the applicability of the factors of in- 
struction, practice and feedback in the context of materials developed to 
teach specific concepts. For purposes of this discussion, we have selected 
the related concepts of '^latitude and longitude" as the vehicle for train- 
ing materials. These concepts are not self-evident to children; that is, 
they must ordinarily be taught formally and they represent concepts of in- 
termediate difficulty for purposes both of teaching and of learning. Fur- 
thermore, and of great importance, they are real concepts, both in the 
sense that they are fundamental to the knowledge and skills of groups such 



as cartographers, geographers and navigators, and in the sense that the 
concepts are customarily found in the social studies or science curri- 
culum studied by children in the middle years of the elementary school. 
Of course, a program of training materials for the teaching of concepts 
would necessarily consist of numerous examples of concepts, perhaps some 
simpler than our illustration, and certainly others of greater complexity. 
An array of the order implied would be required to offer the teacher in 
training sufficient practice to master the skills involved in the teach- 
ing of concepts and to present concepts for practice in which sometimes 
one, sometimes another skill or subset of skills, is stressed. 

Instruct ions . 

The first consideration bears upon how one represents the task to 
the trainee. Based upon what we have reviewed, it seems reasonable to 
infer that the trainee's introduction to the training materials for con- 
cept teaching should involve an overview/ of the nature of concepts .ind 
the elements involved in te^^ching them, Fitts ' observation that skill 
learning begins with a cognitive phase is instructive here. What we wish 
to do is to make it possible for the trainee to see the task in some per- 
spective. This not only helps to guide his learning, but also maximizes 
positive transfer. At the outset, written exposition and demonstrations 
(such as a brief film exemplifying good concept teaching) might assist 
the trainee to cognize the task we wish him to learn. 

Other instructions would probably be desirable and should be em- 
bedded at the beginning of different phases or segments of the training 
material; e.g., prior to the demand to define or illustrate the concept 
to be taught, or the segment in which presenting positive and negative 



examples is to be practiced, and so forth. These verbal instructions 



should be brief and simple, and will probably be helped immensely if 
they are accompanied by good examples of the behavior to be invoked. 

Directions to the teacher in training provide an outline of the 
elements of the training task and an advance statement of classes of 
responses that will be reinforced. The process of training the pro- 
spective teacher in the skill of teaching concepts might begin with a 
direction to him on the order of the following: 

You are going to teach the concepts of "latitude and longitude" 
to a group of 9 and 10 year olds who may be presumed to be of 
average intellectual and educational development. Go home and 
prepare your lesson. Return to this training material twenty- 
four hours from now. The following information and instructions 
will assist you in planning your lesson. 

A. Write down what your students will be able to do after your 
lesson to demonstrate that they have mastered the concepts 
of "latitude" and ''longitude." 

B. Make a list of the subordinate concepts that contribute to 
the concepts you are to teach. For example, the pupil must 
know concepts of direction (North, South, East and West), 
degrees, hemisphere, etc. 

C. For each concept, provide not only a verbal description or 
definition, but a variety of examples or instances of it as 
x-/ell. In the early stages of teaching the concept, your 
examples should all be positive; that is, they should illus- 
trate or exemplify the concept. Come of your later examples 
should be negative (nonexemplars) , particularly if they are 
helpful in differentiating between the concept to be learned 
and another with which it might be confused. 

These or similar instructions are designed to enable the trainee 
to construct a cognitive map of the task of the order that Fitts report- 
ed was useful to fledgling pilots. Another interpretation of sets of 
instructions of the kind provided here is that they function as discrim- 
inative stimuli for the teacher. He is advised in advance, in other 
words, that his efforts to specify master of the concept, or to analyze 



the hierarchy of learnings that contribute to mastery, or to construct 
examples of the concept, will be reinforced (specifically, that they will 
lead to positive knowledge of results, or at least the potential for such 
positive feedback) . 

Practice . 

One of the chief benefits to the prospective teacher of self-contain- 
ed training materials is their provision for both permitting and encourag- 
ing trainees to continue practicing a skill until they have mastered it, 
or until an acceptable criterion has been achieved. How much practice on 
a teaching skill such as the teaching of concepts should the design of a 
program make possible? That, of course, is an empirical question and would 
have to be answered differently for each skill we attempt to teach. As a 
general statement, however, I would urge that you define the amount of 
material available for practice in the most generous terms and then double 
it, perhaps triple it. If there is any single weakness that I think teach- 
er education programs have, beyond their failure to develop penetrating 
concepts about teaching, it is their failure to provide prospective teach- 
ers with an array of skills that are useful for the solution of instruc- 
tional problems. Not only are few skills identified, but it is the rare 
program of teacher education that makes it possible for the candidate to 
achieve the amounts of practice necessary to build a skill adequacy into 
his repertory. If you recall the graphs we showed earlier of continuing 
increments of improvement with large amounts of practice, the point be- 
comes more credible. 

To specify in advance the amount of practice required for any given 
trainee would be impossible and inappropriate, for the aim is to make 
possible whatever an individual trainee needs, and simultaneously to in- 



sure that he engages in practice long enough and productively enough to 
achieve mastery of the skill. This latter is a point of paramount impor- 
tance. Training materials must be constructed with an explicit formula- 
tion in mind of what the trainee must do to demonstrate that he possesses 
mastery of the skill. IVe need not attempt here to delineate standards 
for determining mastery, but they usually involve statements about the 
performance of essential characteristics of a skill and about permissible 
error rates or amount of time within which the skill must be performed. 
These standards should, of course, be presented to the trainee at the 
start of practice and they should be sufficiently clear that the trainee 
himself is as good a judge as anyone of when his performance matches the 
demands of skill mastery. 

For many skills, it is sensible to talk about mastery only when the 
trainee can perform as desired upon a training task that he has not pre- 
viously encountered. This would certainly be true for training in the 
teaching of concepts. For one thing, some of the earliest materials the 
trainee encounters would be too simple to permit mastery in the fullest 
sense. Beyond that, however, the skill we are attempting to build is of 
a general order; that is, the trainee demonstrates the skill when he un- 
dertakes to teach concepts as part of the day to day practice of teaching 
This consideration necessitates, in the case of skills similar to teach- 
ing concepts, that some form of criterion test be available. Those tests 
of course, may simply be represented by the first practice concept the 
trainee teaches within the limits identified for mastery. 

Some years ago, I conducted a research project in which we examined 
the presumed attention-paying behavior of pupils in classes (Hudgins, 
1967) . The point of ^his was to try to relate differences in such atten- 



cion to classroom variables , such as Cype of activity, content, rates of 
activity variation and so forth. One of the daydreams 1 had about extend- 
ing the results of the study to the practical training of teachers con- 
cerned programming our data in such a way that a student teacher would be 
confronted with an immediate problem of the ongoing classroom variety 
and be compelled to construct a "solution*^ to the problem, which would 
be fed into the computer. The computer would then feed back the results 
of the teacher's solution, translated into pupil surface behavior. In 
its most virulent forms, this daydream dictated that the feedback would 
be a visual display, showing the student teacher that his tactic put 
everyone to sleep, had them on the edges of their chairs, and so forth. 
I still think, if one had the right kind of data and concepts, that this 
is a good idea. The value of the concepts to be learned would be para- 
mount, but beyond that, and as a secondary asset, if. such an arrangement 
were available, each student could practice his repertory of skills as 
fully as warranted by his present level of expertness. 

The moral is that teaching skills are made most resistant to for- 
getting when they are overpracticed , that is, when practice continues 
well beyond the stage at which the basic elements of the performance are 
fixed. As you design training materials, it is critical that attention 
be given to ways of insuring that the learner can build his practice 
schedule in terms of the needs of his level of skill and his motivational 
state. The more he must rely upon coordination of others' time, involve- 
ment, the reservation of equipment or facilities, the more impediments 
are built into his practice. These elements combine geometrically, so 
that by the time practice is made contingent upon bringing together two 
or three outside resources, you have virtually insured that no practice 
will occur. 



This consideration, in my opinion, so far overshadows issues such 
as whole vs. part practice, or massed vs. distributed practice, that you 
ought only to consider their pertinence when you are satisfied that the 
necessary prerequisites for self-determined practice schedules are avail- 
able. 

Let us stop here to anticipate an objection to the line of argument 
we are building. The thesis is that teachers receive relatively little 
guided practice in learning skills of teaching and that self-contained 
training materials can go a long way toward reducing the negative conse- 
quences of that fact. Obviously, this general principle is well known 
in many daily fields apart from training materials in teacher education. 
Several that come to mind, for instance, -are practice on a musical instru 
ment and numerous motor skills of the kinds involved in athletics. The 
pitcher may practice for hours until he can invariably hit a small bulls- 
eye from 60 '6"; the batter enthusiastically hammers baseballs thrown his 
way by the tireless mechanical pitcher and so forth. But as we have al- 
ready suggested, college students and others who aspire to teach are or- 
dinarily able to acquire little in the way of directed practice in teach- 
ing, and that little frequently occurs around rather general tasks, such 
as "teaching a whole group," "leading a reading group," etc., instead 
of the precisely defined tasks of teaching made possible by the design 
of training materials . 

A reasonable question to raise is whether teaching can be conducted 
in a meaningful way in the absence of some kind of audience cast in the 
role of pupils or learners. That is, one might agree with everything we 
have said from the viewpoint of logistics but feel compelled to disagree 
in a fundamental sense that skills in teaching can be developed in the 
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abtience of pupils. It may be useful to distinguish between learning the 
principles of a ski Lied perfonranco and learning to perform the skill in 
the context in whicii it normally is practiced. Self-contained trriining 
materials make possible the first sort of practice but net the latter. 
A legitimate and useful role exists for self-contained training materials 
provided that we are cognizant of this distinction and do not erroneously 
attribute the effects of context-based practice to it. It is quite clear 
that trainees who practice the teaching of concepts with self-contained 
materials must at least exhibit mastery of the skill in a contextual set- 
ting and may require some practice in such a setting before they are able 
to apply the principles learned in their earlier practice to more realis- 
tic situations containing, as they do, "warts and all-" It is an empiri- 
cal question as to how comparable are units of self-contained practice 
with units of what we' have called context-based practice. We would hypo- 
thesize that the latter form of practice possesses little if any advan- 
tage over the former, but even if that contention cannot be supported, the 
earlier ventured benefits of self-contained practice lend it a great ad- 
vantage. In the final analysis, the amount of time devoted to practice 
is less critical than the accessibility of the trainee to meaningful prac- 
tice. To put it practically, a student can achieve mastery of a teaching 
skill if he can practice it with the same ease he can read a book in the 
library, in a fraction of the elapsed time required for complex practice 
arrangements even though the learning curve is more sharply accelerated 
in the latter situation. 

A related question concerns the mode or format of practice. Self- 
contained materials will often be remarkably simple and, although they 
must require responses from the trainee that are cognitively identifiable 
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with the responses that are made in performing the skilJ in actual teach- 
ing situations, Lhu mode of rorfponding will typically be very dif Cerent 
and very much simpler. For example, the trainee may construct a written 
response which is then compared with a predesigned array of potential res- 
ponses , and he may respond much as he would to multiple choice items in 
an examination or in programmed instructional materials. Once again, the 
hypothesis is that the trainee, learns generalizations and basic principles 
about the skill which will transfer positively and in large measure to 
context-based practice although to be sure it must be demonstrated that 
the practical exigencies of the teaching situation can be mastered by stu- 
dents so trained. 

Knowledge of results . 

As necessary as practice is to the development of teaching skills, 
its impact can be greatly intensified if the design of the training mate- 
rial builds in appropriate types of information about performance. It is 
common to say that teaching performance suffers from an absence of feed- 
back, but 1 disagree with that. I believe one of the difficulties with 
changing the behavior of experienced teachers is that they receive too 
much of the wrong- kind of knowledge of results. Most of the feedback 
teachers receive is \^hat Holding (1965) would call action feedback rather 
than l_earninfi feedback. To put it a little differently, I think teachers 
receive feedback about style rather than about learning. Time and again, 
when we ask teachers how they know whether a lesson goes well or poorly, 
the responses are the same. "When the kids wave their hands, when they're 
halfway out of their seats, when they^re popping with answers and ideas, 
then you know it's going well.'' We all like the feelings of success that 
such feedback provides, but it is doubtful that the informational value 



for shaping the fine-grained behavior required in excellenc teaching is 
provided by it. 

\'Jhat can we say about knowledge of results that would be of such in- 
formational value in training materials? Of course, the single most im- 
portant decision is the one to insure that feedback is provided as the 
learner practices his skill. The difficulty, as we have already tried to 
clarify, is that most teaching performances have very little in the way o 
intrinsic cues to provide feedback. Thus, the trainer is going to have t 
depend heavily upon augmented or artificial knowledge of results. 

In the early stages of learning to teach concepts, for example, we 
might wish to insist that the trainee always assess the pre-ins tructional 
knowledge about the target concept of his subjects. If he fails to do so 
there may be a signal that brings this fact to his attention. In a real 
teaching situation, of course, it is unlikely that any such signal would 
be available. Similarly, one needs to invest patience and a degree of 
divergent thinking in concept teaching at least at two points: when pro- 
viding positive and negat've examples and when arraying interfering or 
overlapping concepts which are to be differentiated from the target con- 
cept. Some of the best concept teaching episodes arise when students 
become involved in questioning examples, in inquiring whether X or Y 
would also be an example, or in challenging the teacher's assertion that 
A is similar to the target concept in these ways and dissimilar in others 
Such questions and challenges from pupils have a disciplining effect on 
the teacher's behavior, forcing him to engage in the fine grain of con- 
cept teaching which is absolutely critical to its being done competently. 
One cannot depend, of course, on monitoring of that sort occuring on the 
part of pupils. In training, at least, the would-be teacher must receive 
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feedback that tells him not only whether he has engaged the basic elements 
of the skill, but also how effectively he has done so. 

With skill training that is model based, divergent responses by train- 
ee., are both inevitable and desirable. How the developer can prearrange 
feedback that will be useful and meaningful under those circumstances poses 
a very difficult problem. However, it is a problem exactly analogous to 
that faced some years ago by psychologists engaged in constructing tests 
of creative thinking. By definitions, of course, creative responses are 
divergent responses. The work by Torrance (1966) on this score is instruc-- 
tive. The scoring manual for his tests of creative thinking includes some 
general conception of responses that are allowable for the various factors 
scored (for example, originality) but, in addition, the manual contains 
several pages of specific responses that would be given originality scores 
and of responses that are not to be counted as original. True, the Torrance 
Keys are constructed on the basis of responses of a large sample of indivi- 
duals who took the Torrance Tests of Creative Thinking and, thus, the key 
has an empirical basis. This procedure is probably closer to what could 
be done in Stage 2a of our diagram. On the other hand, even at Stage 2, 
responses will be generated for the various elements of the model. The 
problem for the developer is to produce exhaustive responses generated by 
the model and to manage their ordering and display in such a way that they 
can provide appropriate feedback to the trainee. Again, this is a diffi- 
cult problem, but the example of Torrance and the scoring of tests of 
creative thinking suggest that it is by no means an insuperable one. 

It is helpful, I hope, to have some possibly different ideas about 
considerations in the design of training materials, even though the prac- 
tical problems presented are awesome. For I think we will be willing to 
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make che investment of time and energy in designing and producing train- 
ing materials if there is some hope that they will ultimately do their 
task more effectively. 
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